Efficient allocation of network resources, such as available network bandwidth, has become critical as enterprises increase reliance on distributed computing environments and wide area computer networks to accomplish critical tasks. The widely-used TCP/IP protocol suite, which implements the world-wide data communications network environment called the Internet and is employed in many local area networks, omits any explicit supervisory function over the rate of data transport over the various devices that comprise the network. While there are certain perceived advantages, this characteristic has the consequence of juxtaposing very high-speed packets and very low-speed packets in potential conflict and produces certain inefficiencies. Certain loading conditions degrade performance of networked applications and can even cause instabilities which could lead to overloads that could stop data transfer temporarily.
In order to understand the context of certain embodiments of the invention, the following provides an explanation of certain technical aspects of a packet based telecommunications network environment. Internet/Intranet technology is based largely on the TCP/IP protocol suite. At the network level, IP provides a “datagram” delivery service that is, IP is a protocol allowing for delivery of a datagram or packet between two hosts. By contrast, TCP provides a transport level service on top of the datagram service allowing for guaranteed delivery of a byte stream between two IP hosts. In other words, TCP is responsible for ensuring at the transmitting host that message data is divided into packets to be sent, and for reassembling, at the receiving host, the packets back into the complete message.
TCP has “flow control” mechanisms operative at the end stations only to limit the rate at which a TCP endpoint will emit data, but it does not employ explicit data rate control. The basic flow control mechanism is a “sliding window”, a window which by its sliding operation essentially limits the amount of unacknowledged transmit data that a transmitter is allowed to emit. Another flow control mechanism is a congestion window, which is a refinement of the sliding window scheme involving a conservative expansion to make use of the full, allowable window.
The sliding window flow control mechanism works in conjunction with the Retransmit Timeout Mechanism (RTO), which is a timeout to prompt a retransmission of unacknowledged data. The timeout length is based on a running average of the Round Trip Time (RTT) for acknowledgment receipt, i.e. if an acknowledgment is not received within (typically) the smoothed RTT+4*mean deviation, then packet loss is inferred and the data pending acknowledgment is re-transmitted. Data rate flow control mechanisms which are operative end-to-end without explicit data rate control draw a strong inference of congestion from packet loss (inferred, typically, by RTO). TCP end systems, for example, will “back-off,”—i.e., inhibit transmission in increasing multiples of the base RTT average as a reaction to consecutive packet loss.
A crude form of bandwidth management in TCP/IP networks (that is, policies operable to allocate available bandwidth from a single logical link to network flows) is accomplished by a combination of TCP end systems and routers which queue packets and discard packets when some congestion threshold is exceeded. The discarded and therefore unacknowledged packet serves as a feedback mechanism to the TCP transmitter. Routers support various queuing options to provide for some level of bandwidth management. These options generally provide a rough ability to partition and prioritize separate classes of traffic. However, configuring these queuing options with any precision or without side effects is in fact very difficult, and in some cases, not possible. Seemingly simple things, such as the length of the queue, have a profound effect on traffic characteristics. Discarding packets as a feedback mechanism to TCP end systems may cause large, uneven delays perceptible to interactive users. Moreover, while routers can slow down inbound network traffic by dropping packets as a feedback mechanism to a TCP transmitter, this method often results in retransmission of data packets, wasting network traffic and, especially, inbound capacity of a WAN link. In addition, routers can only explicitly control outbound traffic and cannot prevent inbound traffic from over-utilizing a WAN link. A 5% load or less on outbound traffic can correspond to a 100% load on inbound traffic, due to the typical imbalance between an outbound stream of acknowledgments and an inbound stream of data.
In response, certain data flow rate control mechanisms have been developed to provide a means to control and optimize efficiency of data transfer as well as allocate available bandwidth among a variety of business enterprise functionalities. For example, U.S. Pat. No. 6,038,216 discloses a method for explicit data rate control in a packet-based network environment without data rate supervision. Data rate control directly moderates the rate of data transmission from a sending host, resulting in just-in-time data transmission to control inbound traffic and reduce the inefficiencies associated with dropped packets. Bandwidth management devices allow for explicit data rate control for flows associated with a particular traffic classification. For example, U.S. Pat. No. 6,412,000, above, discloses automatic classification of network traffic for use in connection with bandwidth allocation mechanisms. U.S. Pat. No. 6,046,980 discloses systems and methods allowing for application layer control of bandwidth utilization in packet-based computer networks. For example, bandwidth management devices allow network administrators to specify policies operative to control and/or prioritize the bandwidth allocated to individual data flows according to traffic classifications. In addition, certain bandwidth management devices, as well as certain routers, allow network administrators to specify aggregate bandwidth utilization controls to divide available bandwidth into partitions. With some network devices, these partitions can be configured to ensure a minimum bandwidth and/or cap bandwidth as to a particular class of traffic. An administrator specifies a traffic class (such as FTP data, a subnet, or data flows involving a specific user, etc.) and the size of the reserved virtual link—i.e., minimum guaranteed bandwidth and/or maximum bandwidth. Such partitions can be applied on a per-application basis (protecting and/or capping bandwidth for all traffic associated with an application) or a per-user basis (protecting and/or capping bandwidth for a particular user). In addition, certain bandwidth management devices allow administrators to define a partition hierarchy by configuring one or more partitions dividing the access link and further dividing the parent partitions into one or more child partitions.
To facilitate the implementation, configuration and management tasks associated with bandwidth management and other network devices including traffic classification functionality, various traffic classification configuration models and data structures have been implemented. For example, various routers allow network administrators to configure access control lists (ACLs) consisting of an ordered set of access control entries (ACEs). Each ACE contains a number of fields that are matched against the attributes of a packet entering or exiting a given interface. In addition, each ACE has an associated action that indicates what the routing system should do with the packet when a match occurs. ACLs can be configured to accomplish or facilitate a variety of tasks, such as security, redirection, caching, encryption, network address translation, and policy routing. Once configured by an administrator, the routing system compiles the ACL into a hash table to expedite the Look up process during operation of the system.
In addition, U.S. Pat. No. 6,412,000 discloses methods and system that automatically classify network traffic according to a set of classification attributes. As this application teaches, the traffic classification configuration can be arranged in a hierarchy, where classification of a particular packet or data flow traverses a network traffic classification tree until a matching leaf traffic class, if any, is found. Such prior art classification trees are data structures reflecting the hierarchical aspect of traffic class relationships, wherein each node of the tree represents a traffic class and includes a set of attributes or matching rules characterizing the traffic class. The traffic classification, at each level of the hierarchy, determines whether the data flow or packet matches the attributes of a given traffic class node and, if so, continues the process for child traffic class nodes down to the leaf nodes. In certain modes, unmatched data flows map to a default traffic class. In addition, patent application Ser. No. 10/039,992 discloses methods for caching portions of hierarchical classification trees in hash tables to optimize traffic classification lookups.
In addition, many enterprises have implemented Virtual Private Networks (VPNs) to provide individual users and/or branch offices secure remote access to the resources available over the respective enterprises' networks (such as a data center). A VPN provides the ability to use a public telecommunication infrastructure, such as the Internet, to provide remote offices or individual users with secure access to their organization's network. A VPN works by using the shared public infrastructure while maintaining privacy through security procedures and tunneling protocols such as the Layer Two Tunneling Protocol (L2TP). In effect, the protocols, by encrypting data at the sending end and decrypting it at the receiving end, send the data through a “tunnel” (achieved by encapsulating TCP packets comprising the data within VPN headers) that cannot be “entered” by data that is not property encrypted. An additional level of security involves encrypting not only the data, but also the originating and receiving computer network addresses. A typical VPN deployment involves a VPN server deployed on an enterprise's network and at least one VPN client remote from the enterprise network.
Beyond security concerns, enterprises may also use tunnels to optimize network traffic. Specifically, tunnel technologies can be implemented to improve network performance of a communications path. For example, data compression and other technologies that optimize network traffic can be deployed to improve the efficiency and performance of a computer network and ease congestion at bottleneck links. For example, implementing data compression and/or caching technology can improve network performance by reducing the amount of bandwidth required to transmit a given block of data between two network devices along a communications path. Data compression technologies can be implemented on routing nodes without alteration of client or server end systems, or software applications executed therein, to reduce bandwidth requirements along particularly congested portions of a communications path. For example, tunnel technologies, like those used in Virtual Private Network (VPN) implementations, establish tunnels through which network traffic is transformed upon entering at a first network device in a communications path and restored to substantially the same state upon leaving a second network device in the communications path.
Given such tunnel technologies, the network traffic across a particular access link can be, and often is, a combination of tunneled data flows (e.g., encapsulated and/or transformed (encrypted, compressed, etc.) data flows and non-tunneled data flows. The transformation of data flows involved in such tunnels, however, does present certain problems if a network administrator desires to manage bandwidth utilization associated with network traffic within such tunnels and outside such tunnels. As discussed above, certain bandwidth management solutions allow for classification of network traffic on an application-level basis. The transformation of data flows, however, severely impacts the ability of such bandwidth management devices to classify such data flows as the encryption or other transformation obscures the higher layer information in the data flows, and in many instances, the IP addresses of the actual source and destination hosts. Accordingly, while prior art bandwidth management devices can be configured to detect tunneled traffic, they cannot concurrently classify non-tunneled and tunneled network traffic. Moreover, the transformation of data flows, such as compression, renders it difficult to intelligently apply bandwidth utilization controls to tunneled traffic. For example, a compressed data flow may comprise 75 Kbytes of data, for example, while the de-compressed data flow may comprise 100 Kbytes. While a bandwidth management device located on the outside of a tunnel is able to adequately classify the data flow and quantify the size of the data flow associated with the de-compressed network traffic, it has no visibility into the compressed data flow and its actual impact on bandwidth utilization across an access link. The resulting lack of visibility into the actual impact of the data flow renders it difficult to intelligently apply bandwidth utilization controls (such as pacing an inbound data flow to control aggregate bandwidth utilization) in a manner that responds to the actual impact of the data flow. Similarly, for the reasons discussed above, a bandwidth management device located within the tunnel path has visibility into the actual impact of the compressed data flow, but does not have the ability to adequately classify the tunneled network traffic on an application-level or other basis.
In light of the foregoing, a need in the art exists for methods, apparatuses and systems that facilitate concurrent classification and control of tunneled and non-tunneled network traffic. Embodiments of the present invention substantially fulfill this need.